I'm following the kOps tutorial to set up a cluster on AWS. I am able to create a cluster with
kops create cluster
kops update cluster --yes
However, when validating whether my cluster is set up correctly with
kops validate cluster
I get stuck with error:
unexpected error during validation: error listing nodes: Unauthorized
The same error happens in many other kOps operations.
I checked my kOps/K8s version and it is 1.19:
> kops version
Version 1.19.1 (git-8589b4d157a9cb05c54e320c77b0724c4dd094b2)
> kubectl version
Client Version: version.Info{Major:"1", Minor:"20" ...
Server Version: version.Info{Major:"1", Minor:"19" ...
How can I fix this?
As of kOps 1.19 there are two reasons you will suddenly get this error:
Both issues above are fixed by running kops export kubecfg --admin.
Note that using the default TLS credentials is discouraged. Consider things like using an OIDC provider instead.
Kubernetes v1.19 removed basic auth support, incidentally making the default kOps credentials unable to authorize. To work around this, we will update our cluster to use a Network Load Balancer (NLB) instead of the default Classic Load Balancer (CLB). The NLB can be accessed with non-deprecated AuthZ mechanisms.
After creating your cluster, but before updating cloud resources (before running with --yes), edit its configuration to use a NLB:
kops edit cluster
Then update your load balancer class to Network:
spec:
api:
loadBalancer:
class: Network
Now update cloud resources with
kops update cluster --yes
And you'll be able to pass AuthZ with kOps on your cluster.
Note that there are several other advantages to using an NLB as well, check the AWS docs for a comparison.
If you have a pre-existing cluster you want to update to a NLB, there are more steps to follow to ensure clients don't start failing AuthZ, to delete old resources, etc. You'll find a better guide for that in the kOps v1.19 release notes.